Temporal prediction of parameters in non-linear adaptive loop filter

ABSTRACT

Devices, systems and methods for temporal prediction of parameters in non-linear adaptive loop filtering are described. In an exemplary aspect, a method for visual media processing includes configuring, for a current video block, one or more parameters of a clipping operation that is part of a non-linear filtering operation; and performing, based on the one or more parameters, a conversion between the current video block and a bitstream representation of the current video block, wherein the one or more parameters is coded in accordance with a rule.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/CN2020/084876, filed on Apr. 15, 2020, which claims the priorityto and benefits of International Patent Application No.PCT/CN2019/082626, filed on Apr. 15, 2019. All the aforementioned patentapplications are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

This patent document relates to video coding and decoding techniques,devices and systems.

BACKGROUND

In spite of the advances in video compression, digital video stillaccounts for the largest bandwidth use on the internet and other digitalcommunication networks. As the number of connected user devices capableof receiving and displaying video increases, it is expected that thebandwidth demand for digital video usage will continue to grow.

SUMMARY

Devices, systems and methods related to digital video coding, andspecifically, to temporal prediction of parameters in non-linearadaptive loop filtering are described. The described methods may beapplied to both the existing video coding standards (e.g., HighEfficiency Video Coding (HEVC)) and future video coding standards (e.g.,Versatile Video Coding (VVC)) or codecs.

In one representative aspect, the disclosed technology may be used toprovide a method for visual media processing. This method includesconfiguring, for a current video block, one or more parameters of aclipping operation that is part of a non-linear filtering operation; andperforming, based on the one or more parameters, a conversion betweenthe current video block and a bitstream representation of the currentvideo block, wherein the one or more parameters is coded in accordancewith a rule.

In another representative aspect, the disclosed technology may be usedto provide a method for visual media processing. This method includesdetermining, based on a characteristic of a current video block, one ormore parameters of a non-linear filtering operation; and performing,based on the one or more parameters, a conversion between the currentvideo block and a bitstream representation of the current video block.

In yet another representative aspect, the disclosed technology may beused to provide a method for visual media processing. This methodincludes configuring, for a current video block, one or more parametersof a clipping operation that is part of a non-linear filteringoperation; and performing, based on the one or more parameters, aconversion between the current video block and a bitstreamrepresentation of the current video block, wherein the one or moreparameters are presented in the bitstream representation independentlyfrom values of at least one filter coefficient associated with thenon-linear filtering operation.

In yet another representative aspect, the disclosed technology may beused to provide a method for visual media processing. This methodincludes configuring, for a current video block, one or more parametersof a clipping operation that is part of a non-linear filteringoperation; and performing, based on the one or more parameters, aconversion between the current video block and a bitstreamrepresentation of the current video block, wherein the current videoblock inherits filter coefficients from an i-th filter, and wherein afirst rule associated with inheritance of the one or more parameters ofthe clipping operation is different from a second rule associated withinheritance of the filter coefficients.

In yet another representative aspect, the above-described method isembodied in the form of processor-executable code and stored in acomputer-readable program medium.

In yet another representative aspect, a video encoder apparatus mayimplement a method as described herein.

In yet another representative aspect, a video decoder apparatus mayimplement a method as described herein.

The above and other aspects and features of the disclosed technology aredescribed in greater detail in the drawings, the description and theclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of an encoder block diagram for video coding.

FIGS. 2A, 2B and 2C show examples of geometry transformation-basedadaptive loop filter (GALF) filter shapes.

FIG. 3 shows an example of a flow graph for a GALF encoder decision.

FIGS. 4A-4D show example subsampled Laplacian calculations for adaptiveloop filter (ALF) classification.

FIG. 5 shows an example of neighboring samples utilized in a bilateralfilter.

FIG. 6 shows an example of windows covering two samples utilized in aweight calculation.

FIG. 7 shows an example of a scan pattern.

FIGS. 8A-8C show flowcharts of exemplary methods for the temporalprediction of parameters in non-linear adaptive loop filtering.

FIG. 9 is a block diagram of an example of a hardware platform forimplementing a video decoding or a video encoding technique described inthe present document.

FIG. 10 is a block diagram of an example video processing system inwhich disclosed techniques may be implemented.

FIG. 11 shows a flowchart of an example method for visual mediaprocessing.

FIG. 12 shows a flowchart of an example method for visual mediaprocessing.

FIG. 13 shows a flowchart of an example method for visual mediaprocessing.

FIG. 14 shows a flowchart of an example method for visual mediaprocessing.

DETAILED DESCRIPTION

Due to the increasing demand of higher resolution video, video codingmethods and techniques are ubiquitous in modern technology. Video codecstypically include an electronic circuit or software that compresses ordecompresses digital video, and are continually being improved toprovide higher coding efficiency. A video codec converts uncompressedvideo to a compressed format or vice versa. There are complexrelationships between the video quality, the amount of data used torepresent the video (determined by the bit rate), the complexity of theencoding and decoding algorithms, sensitivity to data losses and errors,ease of editing, random access, and end-to-end delay (latency). Thecompressed format usually conforms to a standard video compressionspecification, e.g., the High Efficiency Video Coding (HEVC) standard(also known as H.265 or MPEG-H Part 2), the Versatile Video Coding (VVC)standard to be finalized, or other current and/or future video codingstandards.

Video coding standards have evolved primarily through the development ofthe well-known ITU-T and ISO/IEC standards. The ITU-T produced H.261 andH.263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and the twoorganizations jointly produced the H.262/MPEG-2 Video and H.264/MPEG-4Advanced Video Coding (AVC) and H.265/HEVC standards. Since H.262, thevideo coding standards are based on the hybrid video coding structurewherein temporal prediction plus transform coding are utilized. Toexplore the future video coding technologies beyond HEVC, Joint VideoExploration Team (JVET) was founded by VCEG and MPEG jointly in 2015.Since then, many new methods have been adopted by JVET and put into thereference software named Joint Exploration Model (JEM). In April 2018,the Joint Video Expert Team (JVET) between VCEG (Q6/16) and ISO/IEC JTC1SC29/WG11 (MPEG) was created to work on the VVC standard targeting at50% bitrate reduction compared to HEVC.

Embodiments of the disclosed technology may be applied to existing videocoding standards (e.g., HEVC, H.265) and future standards to improveruntime performance. Section headings are used in the present documentto improve readability of the description and do not in any way limitthe discussion or the embodiments (and/or implementations) to therespective sections only.

1 Examples of Color Space and Chroma Subsampling

Color space, also known as the color model (or color system), is anabstract mathematical model which simply describes the range of colorsas tuples of numbers, typically as 3 or 4 values or color components(e.g. RGB). Basically speaking, color space is an elaboration of thecoordinate system and sub-space.

For video compression, the most frequently used color spaces are YCbCrand RGB.

YCbCr, Y′CbCr, or Y Pb/Cb Pr/Cr, also written as YCBCR or Y′CBCR, is afamily of color spaces used as a part of the color image pipeline invideo and digital photography systems. Y′ is the luma component and CBand CR are the blue-difference and red-difference chroma components. Y′(with prime) is distinguished from Y, which is luminance, meaning thatlight intensity is nonlinearly encoded based on gamma corrected RGBprimaries.

Chroma subsampling is the practice of encoding images by implementingless resolution for chroma information than for luma information, takingadvantage of the human visual system's lower acuity for colordifferences than for luminance.

1.1 The 4:4:4 Color Format

Each of the three Y′CbCr components have the same sample rate, thusthere is no chroma subsampling. This scheme is sometimes used inhigh-end film scanners and cinematic post production.

1.2 The 4:2:2 Color Format

The two chroma components are sampled at half the sample rate of luma,e.g. the horizontal chroma resolution is halved. This reduces thebandwidth of an uncompressed video signal by one-third with little to novisual difference.

1.3 The 4:2:0 Color Format

In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but asthe Cb and Cr channels are only sampled on each alternate line in thisscheme, the vertical resolution is halved. The data rate is thus thesame. Cb and Cr are each subsampled at a factor of 2 both horizontallyand vertically. There are three variants of 4:2:0 schemes, havingdifferent horizontal and vertical siting.

-   -   In MPEG-2, Cb and Cr are cosited horizontally. Cb and Cr are        sited between pixels in the vertical direction (sited        interstitially).    -   In JPEG/JFIF, H.261, and MPEG-1, Cb and Cr are sited        interstitially, halfway between alternate luma samples.    -   In 4:2:0 DV, Cb and Cr are co-sited in the horizontal direction.        In the vertical direction, they are co-sited on alternating        lines.

2 Examples of the Coding Flow of a Typical Video Codec

FIG. 1 shows an example of encoder block diagram of VVC, which containsthree in-loop filtering blocks: deblocking filter (DF), sample adaptiveoffset (SAO) and ALF. Unlike DF, which uses predefined filters, SAO andALF utilize the original samples of the current picture to reduce themean square errors between the original samples and the reconstructedsamples by adding an offset and by applying a finite impulse response(FIR) filter, respectively, with coded side information signaling theoffsets and filter coefficients. ALF is located at the last processingstage of each picture and can be regarded as a tool trying to catch andfix artifacts created by the previous stages.

3 Examples of a Geometry Transformation-Based Adaptive Loop Filter inJEM

In the JEM, an geometry transformation-based adaptive loop filter (GALF)with block-based filter adaption is applied. For the luma component, oneamong 25 filters is selected for each 2×2 block, based on the directionand activity of local gradients.

3.1 Examples of Filter Shape

In the JEM, up to three diamond filter shapes (as shown in FIGS. 2A, 2Band 2C for the 5×5 diamond, 7×7 diamond and 9×9 diamond, respectively)can be selected for the luma component. An index is signalled at thepicture level to indicate the filter shape used for the luma component.For chroma components in a picture, the 5×5 diamond shape is alwaysused.

3.1.1 Block Classification Each 2×2 block is categorized into one out of25 classes. The classification index C is derived based on itsdirectionality D and a quantized value of activity Â, as follows:

C=5D+Â.  (1)

To calculate D and Â, gradients of the horizontal, vertical and twodiagonal direction are first calculated using 1-D Laplacian:

$\begin{matrix}{{g_{v} = {\sum\limits_{k = {i - 2}}^{i + 3}{\sum\limits_{l = {j - 2}}^{j + 3}V_{k,l}}}},{V_{k,l} = \left| {{2{R\left( {k,\ l} \right)}} - {R\left( {k,\ {l - 1}} \right)} - {R\left( {k,\ {l + 1}} \right)}} \right|},} & (2) \\{{g_{h} = {\sum\limits_{k = {i - 2}}^{i + 3}{\sum\limits_{l = {j - 2}}^{j + 3}H_{k,l}}}},{H_{k,l} = \left| {{2{R\left( {k,\ l} \right)}} - {R\left( {{k - 1},\ l} \right)} - {R\left( {{k + 1},\ l} \right)}} \right|},} & (3) \\{{g_{d1} = {\sum\limits_{k = {i - 2}}^{i + 3}{\sum\limits_{l = {j - 3}}^{j + 3}{D1_{k,l}}}}},{{D\; 1_{k,l}} = \left| {{2{R\left( {k,\ l} \right)}} - {R\left( {{k - 1},\ {l - 1}} \right)} - {R\left( {{k + 1},\ {l + 1}} \right)}} \right|}} & (4) \\{{g_{d2} = {\sum\limits_{k = {i - 2}}^{i + 3}{\sum\limits_{j = {j - 2}}^{j + 3}{D2_{k,l}}}}},{{D\; 2_{k,l}} = \left| {{2{R\left( {k,\ l} \right)}} - {R\left( {{k - 1},\ {l + 1}} \right)} - {R\left( {{k + 1},\ {l - 1}} \right)}} \right|}} & (5)\end{matrix}$

Indices i and j refer to the coordinates of the upper left sample in the2×2 block and R(i, j) indicates a reconstructed sample at coordinate(i,j).Then D maximum and minimum values of the gradients of horizontal andvertical directions are set as:

g _(h,v) ^(max)=max(g _(h) ,g _(v)), g _(h,v) ^(min)=min(g _(h) ,g_(v)),  (6)

and the maximum and minimum values of the gradient of two diagonaldirections are set as:

g _(d0,d1) ^(max)=max(g _(d0) ,g _(d1)), g _(d0,d1) ^(min)=min(g _(d0),g _(d1)),  (7)

To derive the value of the directionality D, these values are comparedagainst each other and with two thresholds t₁ and t₂:Step 1. If both g_(h,v) ^(max)≤t₁·g_(h,v) ^(min) and g_(d0,d1)^(max)≤t₁·g_(d0,d1) ^(min) are true, D is set to 0.Step 2. If g_(h,v) ^(max)/g_(h,v) ^(min)>g_(d0,d1) ^(max)/g_(d0,d1)^(min), continue from Step 3; otherwise continue from Step 4.Step 3. If g_(h,v) ^(max)>t₂·g_(h,v) ^(min), is set to 2; otherwise D isset to 1.Step 4. If g_(d0,d1) ^(max)>t₂·g_(d0,d1) ^(min), is set to 4; otherwiseD is set to 3.The activity value A is calculated as:

$\begin{matrix}{{A = {\sum\limits_{k = {i - 2}}^{i + 3}{\sum\limits_{l = {j - 2}}^{j + 3}\left( {V_{k,l} + H_{k,l}} \right)}}}.} & (8)\end{matrix}$

A is further quantized to the range of 0 to 4, inclusively, and thequantized value is denoted as Â. For both chroma components in apicture, no classification method is applied, i.e. a single set of ALFcoefficients is applied for each chroma component.

3.1.2 Geometric Transformations of Filter Coefficients

Before filtering each 2×2 block, geometric transformations such asrotation or diagonal and vertical flipping are applied to the filtercoefficients f(k, l) depending on gradient values calculated for thatblock. This is equivalent to applying these transformations to thesamples in the filter support region. The idea is to make differentblocks to which ALF is applied more similar by aligning theirdirectionality.

Three geometric transformations, including diagonal, vertical flip androtation are introduced:

Diagonal: f _(D)(k,l)=f(l,k),

Vertical flip: f _(V)(k,l)=f(k,K−l−1),

Rotation: f _(R)(k,l)=f(K−l−1,k).  (9)

Herein, K is the size of the filter and 0≤k, l≤K−1 are coefficientscoordinates, such that location (0,0) is at the upper left corner andlocation (K−1, K−1) is at the lower right corner. The transformationsare applied to the filter coefficients f(k, l) depending on gradientvalues calculated for that block. The relationship between thetransformation and the four gradients of the four directions aresummarized in Table 1.

TABLE 1 Mapping of the gradient calculated for one block and thetransformations Gradient values Transformation g_(d2) < g_(d1) and g_(h)< g_(v) No transformation g_(d2) < g_(d1) and g_(v) < g_(h) Diagonalg_(d1) < g_(d2) and g_(h) < g_(v) Vertical flip g_(d1) < g_(d2) andg_(v) < g_(h) Rotation

3.1.3 Signaling of Filter Parameters

In the JEM, GALF filter parameters are signaled for the first CTU, i.e.,after the slice header and before the SAO parameters of the first CTU.Up to 25 sets of luma filter coefficients could be signaled. To reducebits overhead, filter coefficients of different classification can bemerged. Also, the GALF coefficients of reference pictures are stored andallowed to be reused as GALF coefficients of a current picture. Thecurrent picture may choose to use GALF coefficients stored for thereference pictures, and bypass the GALF coefficients signaling. In thiscase, only an index to one of the reference pictures is signaled, andthe stored GALF coefficients of the indicated reference picture areinherited for the current picture.

To support GALF temporal prediction, a candidate list of GALF filtersets is maintained. At the beginning of decoding a new sequence, thecandidate list is empty. After decoding one picture, the correspondingset of filters may be added to the candidate list. Once the size of thecandidate list reaches the maximum allowed value (i.e., 6 in currentJEM), a new set of filters overwrites the oldest set in decoding order,and that is, first-in-first-out (FIFO) rule is applied to update thecandidate list. To avoid duplications, a set could only be added to thelist when the corresponding picture doesn't use GALF temporalprediction. To support temporal scalability, there are multiplecandidate lists of filter sets, and each candidate list is associatedwith a temporal layer. More specifically, each array assigned bytemporal layer index (TempIdx) may compose filter sets of previouslydecoded pictures with equal to lower TempIdx. For example, the k-tharray is assigned to be associated with TempIdx equal to k, and it onlycontains filter sets from pictures with TempIdx smaller than or equal tok. After coding a certain picture, the filter sets associated with thepicture will be used to update those arrays associated with equal orhigher TempIdx.

Temporal prediction of GALF coefficients is used for inter coded framesto minimize signaling overhead. For intra frames, temporal prediction isnot available, and a set of 16 fixed filters is assigned to each class.To indicate the usage of the fixed filter, a flag for each class issignaled and if required, the index of the chosen fixed filter. Evenwhen the fixed filter is selected for a given class, the coefficients ofthe adaptive filter f(k,l) can still be sent for this class in whichcase the coefficients of the filter which will be applied to thereconstructed image are sum of both sets of coefficients.

The filtering process of luma component can controlled at CU level. Aflag is signaled to indicate whether GALF is applied to the lumacomponent of a CU. For chroma component, whether GALF is applied or notis indicated at picture level only.

3.1.4 Filtering Process

At decoder side, when GALF is enabled for a block, each sample R(i, j)within the block is filtered, resulting in sample value R′(i, j) asshown below, where L denotes filter length, f_(m,n) represents filtercoefficient, and f(k, l) denotes the decoded filter coefficients.

R′(i,j)=Σ_(k=−L/2) ^(L/2)Σ_(l=−L/2) ^(L/2) f(k,l)×R(i+k,j+l)  (10)

3.1.5 Determination Process for Encoder Side Filter Parameters

Overall Encoder Decision Process for GALF is Illustrated in FIG. 3. ForLuma Samples of each CU, the encoder makes a decision on whether or notthe GALF is applied and the appropriate signalling flag is included inthe slice header. For chroma samples, the decision to apply the filteris done based on the picture-level rather than CU-level. Furthermore,chroma GALF for a picture is checked only when luma GALF is enabled forthe picture.

4 Examples of a Geometry Transformation-Based Adaptive Loop Filter inVVC

The current design of GALF in VVC has the following major changescompared to that in JEM:

-   -   1) The adaptive filter shape is removed. Only 7×7 filter shape        is allowed for luma component and 5×5 filter shape is allowed        for chroma component.    -   2) Temporal prediction of ALF parameters and prediction from        fixed filters are both removed.    -   3) For each CTU, one bit flag is signaled whether ALF is enabled        or disabled.    -   4) Calculation of class index is performed in 4×4 level instead        of 2×2. In addition, as proposed in JVET-L0147, sub-sampled        Laplacian calculation method for ALF classification is utilized,        as shown in FIGS. 4A-4D. More specifically, there is no need to        calculate the horizontal/vertical/45 diagonal/135 degree        gradients for each sample within one block. Instead, 1:2        subsampling is utilized.

In VTM4.0, the filtering process of the Adaptive Loop Filter, isperformed as follows:

O(x,y)=Σ_((i,j)) w(i,j)·I(x+i,y+j)  (11)

where samples I(x+i, y+j) are input samples, O(x, y) is the filteredoutput sample (i.e. filter result), and w(i,j) denotes the filtercoefficients. In practice, in VTM4.0 it is implemented using integerarithmetic for fixed point precision computations:

$\begin{matrix}{{O\left( {x,\ y} \right)} = {\left( {{\sum_{i = {- \frac{L}{2}}}^{\frac{L}{2}}{\sum_{j = {- \frac{L}{2}}}^{\frac{L}{2}}{{w\left( {i,j} \right)} \cdot {I\left( {{x + i},{y + j}} \right)}}}} + 64} \right) ⪢ 7}} & (12)\end{matrix}$

where L denotes the filter length, and where w(i,j) are the filtercoefficients in fixed point precision.

5 Non-Linear Adaptive Loop Filtering (ALF) in JVET-N0242 5.1 FilteringReformulation

Equation (11) can be reformulated, without coding efficiency impact, inthe following expression:

O(x,y)=I(x,y)+Σ_((i,j)≠(0,0)) w(i,j)·(I(x+i,y+j)−I(x,y))  (13)

Herein, w(i,j) are the same filter coefficients as in equation (11)[excepted w(0,0) which is equal to 1 in equation (13) while it is equalto 1−Σ_((i,j)≠(0,0))w(i,j) in equation (11)].

5.2 Modified Filter

Using this above filter formula of (13), we can easily introduce nonlinearity to make ALF more efficient by using a simple clipping functionto reduce the impact of neighbor sample values (I(x+i, y+j)) when theyare too different with the current sample value (I(x, y)) beingfiltered.

In this proposal, the ALF filter is modified as follows:

O′(x,y)=I(x,y)+Σ_((i,j)≠(0,0)) w(i,j)×K(I(x+i,y+j)−I(x,y),k(i,j))  (14)

Herein, K(d, b)=min(b, max(−b, d)) is the clipping function, and k(i, j)are clipping parameters, which depends on the(i, j) filter coefficient.The encoder performs the optimization to find the best k(i, j). Notethat for implementation in integer precision, shifting with roundingΣ_((i,j)(0,0))w(i,j)×K(I(x+i, y+j)−I(x, y),k(i, j)) is applied.

In the JVET-N0242 implementation, the clipping parameters k(i,j) arespecified for each ALF filter, one clipping value is signaled per filtercoefficient. It means that up to 12 clipping values can be signalled inthe bitstream per Luma filter and up to 6 clipping values for the Chromafilter.

In order to limit the signaling cost and the encoder complexity, welimit the evaluation of the clipping values to a small set of possiblevalues. In the proposal, we only use 4 fixed values which are the samefor INTER and INTRA tile groups.

Because the variance of the local differences is often higher for Lumathan for Chroma, we use two different sets for the Luma and Chromafilters. We also include the maximum sample value (here 1024 for 10 bitsbit-depth) in each set, so that clipping can be disabled if it is notnecessary.

The sets of clipping values used in the JVET-N0242 tests are provided inthe Table 2. The 4 values have been selected by roughly equallysplitting, in the logarithmic domain, the full range of the samplevalues (coded on 10 bits) for Luma, and the range from 4 to 1024 forChroma.

More precisely, the Luma table of clipping values have been obtained bythe following formula:

${{AlfClip}_{L} = \left\{ {{{round}\mspace{14mu}\left( \left( (M)^{\frac{1}{N}} \right)^{N - n + 1} \right)\mspace{14mu}{for}\mspace{14mu} n} \in {1..N}} \right\rbrack},{{{with}\mspace{14mu} M} = {{2^{10}N} = {4.}}}$

Similarly, the Chroma tables of clipping values is obtained according tothe following formula:

$\left. {{Al{fClip}_{C}} = \left\{ {{{round}\mspace{14mu}\left( {A \cdot \left( \left( \frac{M}{A} \right) \right)^{N - n}} \right)\mspace{14mu}{for}\mspace{14mu} n} \in {1..N}} \right\rbrack} \right\},{{{with}\mspace{14mu} M} = 2^{10}},{N = {{4A} = {4.}}}$

TABLE 2 Authorized clipping values INTRA/INTER tile group LUMA { 1024,181, 32, 6 } CHROMA { 1024, 161, 25, 4 }

The selected clipping values are coded in the “alf_data” syntax elementby using a Golomb encoding scheme corresponding to the index of theclipping value in the above Table 2. This encoding scheme is the same asthe encoding scheme for the filter index.

5.2.1 Syntax, Semantics

The newly introduced syntax changes (shown below in bolded, italicized,and underlined font) due to the NLALF is

7.3.4.3 Adaptive Loop Filter Data Syntax

Descript alf_data( ) { or  alf_chroma_idc tu(v)  alf_luma_clip u(1)  if(alf_choma_idc )   alf_chroma_clip u(1) alf_luma_num_filters_signalled_minus1 tb(v)  if(alf_luma_num_filters_signalled_minus1 > 0) {   for( filtIdx = 0; filtIdx< NumAlfFilters; filtIdx++)    alf_luma_coeff_delta_idx[ filtIdx ] tb(v) }  alf_luma_coeff_delta_flag u(1)  if ( !alf_luma_coeff_delta_flag &&alf_luma_num_filters_signalled_minus1 > 0)  alf_luma_coeff_delta_prediction_flag u(1) alf_luma_min_eg_order_minus1 ue(v)  for( i = 0; i < 3; i++)  alf_luma_eg_order_increase_flag[ i ] u(1)  if (alf_luma_coeff_delta_flag ) {   for( sigFiltIdx = 0; sigFiltIdx <=alf_luma_num_filters_signalled_minus1; sigFiltIdx++)   alf_luma_coeff_flag[ sigFiltIdx ] u(1)  }  for( sigFiltIdx = 0;sigFiltIdx <= alf_luma_num_filters_signalled_minus1; sigFiltIdx++) {  if( alf_luma_coeff_flag[sigFiltIdx ]) {   for ( j = 0; j < 12; j++) {   alf_luma_coeff_delta_abs[sigFiltIdx ][ j ] uek(v)    if(alf_luma_coeff_delta_abs[sigFiltIdx ][ j ])    alf_luma_coeff_delta_sign[sigFiltIdx ][ j ] u(1)    }   }  }  if(alf_luma_clip ) {   alf_luma_clip_min_eg_order_minus1 ue(v)   for( i  = 0;  i  <  3;  i++)    alf_luma_clip_eg_order_increase_flag[ i ] u(1)  for  (  sikFiltIdx  =  0;  sikFiltIdx <=alf_luma_num_filters_siknalled_minus1; sikFiltIdx++)  {    if ( alf_luma_coeff_flag[ sikFiltIdx  ]) {     for(j = 0;  1  < 12; j++ ) {     if( filterCoefficients[ sigFiltIdx ][ j ])      alf_luma_clip_idx[ sigFiltIdx ][ j ] uek(v)     }    }   }  }  if( alf_chroma_idc > 0) {   alf_chroma_min_eg_order_minus1 ue(v)   for( i= 0; i < 2; i++)    alf_chroma_eg_order_increase_flag[ i ] u(1)   for( j= 0; j < 6; j++) {    alf_chroma_coeff_abs[ j ] uek(v)    if(alf_chroma_coeff_abs[ j ] > 0)     alf_chroma_coeff_sign[ j ] u(1)   } }  if ( alf_chroma_idc > 0 && alf_chroma_clip ) {  alf_chroma_clip_min_eg_order_minus1 ue(v)   for( i = 0; i < 2; i++ )   alf_chroma_clip_eg_order_increase_flag[ i ] u(1)   for (  i  =  0;  j <  6; j++ ) {    if (  alf_chroma_coeff_abs[ j ])    alf_chroma_clip_idx[ j ] uek(v)   }  }

6 CTU-Based ALF in JVET-N0427

Adaptive parameter set (APS) was adopted in VTM4. Each APS contains oneset of signalled ALF filters, up to 32 APSs are supported. In theproposal, slice-level temporal filter is tested. A tile group can re-usethe ALF information from an APS to reduce the overhead. The APSs areupdated as a first-in-first-out (FIFO) buffer.

For luma component, when ALF is applied to a luma CTB, the choice among16 fixed, 5 temporal or 1 signaled filter sets (signalled in slicelevel) is indicated. Only the filter set index is signalled. For oneslice, only one new set of 25 filters can be signaled. If a new set issignalled for a slice, all the luma CTBs in the same slice share thatset. Fixed filter sets can be used to predict the new slice-level filterset and can be used as candidate filter sets for a luma CTB as well. Thenumber of filters is 64 in total.

For chroma component, when ALF is applied to a chroma CTB, if a newfilter is signalled for a slice, the CTB used the new filter, otherwise,the most recent temporal chroma filter satisfying the temporalscalability constrain is applied.

As the slice-level temporal filter, the APSs are updated as afirst-in-first-out (FIFO) buffer.

7 Post-Reconstruction Filters 7.1 Diffusion Filter (DF)

In JVET-L0157, diffusion filter is proposed, wherein the intra/interprediction signal of the CU may be further modified by diffusionfilters.

Uniform diffusion filter. The Uniform Diffusion Filter is realized byconvolving the prediction signal with a fixed mask that is either givenas h^(I) or as h^(IV), defined below.

Besides the prediction signal itself, one line of reconstructed samplesleft and above of the block are used as an input for the filteredsignal, where the use of these reconstructed samples can be avoided oninter blocks.

Let pred be the prediction signal on a given block obtained by intra ormotion compensated prediction. In order to handle boundary points forthe filters, the prediction signal needs to be extended to a predictionsignal pred_(ext). This extended prediction can be formed in two ways:

Either, as an intermediate step, one line of reconstructed samples leftand above the block are added to the prediction signal and then theresulting signal is mirrored in all directions. Or only the predictionsignal itself is mirrored in all directions. The latter extension isused for inter blocks. In this case, only the prediction signal itselfcomprises the input for the extended prediction signal pred_(ext).

If the filter h^(I) is to be used, it is proposed to replace theprediction signal pred by

h ^(I)*pred,

using the aforementioned boundary extension. Here, the filter mask h^(I)is given as

$h^{I} = {(0.25)^{4}{\begin{pmatrix}0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 4 & 0 & 4 & 0 & 0 & 0 \\0 & 0 & 6 & 0 & 16 & 0 & 6 & 0 & 0 \\0 & 4 & 0 & 24 & 0 & 24 & 0 & 4 & 0 \\1 & 0 & 16 & 0 & 36 & 0 & 16 & 0 & 1 \\0 & 4 & 0 & 24 & 0 & 24 & 0 & 4 & 0 \\0 & 0 & 6 & 0 & 16 & 0 & 6 & 0 & 0 \\0 & 0 & 0 & 4 & 0 & 4 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0\end{pmatrix}.}}$

If the filter h^(IV) is to be used, it is proposed to replace theprediction signal pred by

h ^(IV)*pred

Here, the filter h^(iv) is given as

h ^(IV) =h ^(I) *h ^(I) *h ^(I) *h ^(I).

Directional diffusion filter. Instead of using signal adaptive diffusionfilters, directional filters, a horizontal filter h^(hor) and a verticalfilter h^(ver), are used which still have a fixed mask. More precisely,the uniform diffusion filtering corresponding to the mask h^(I) of theprevious section is simply restricted to be either applied only alongthe vertical or along the horizontal direction. The vertical filter isrealized by applying the fixed filter mask

$\begin{matrix}{h_{ve\tau} = {\left( {0.5} \right)^{4}\begin{pmatrix}1 \\0 \\4 \\0 \\6 \\0 \\4 \\0 \\1\end{pmatrix}}} & \;\end{matrix}$

to the prediction signal and the horizontal filter is realized by usingthe transposed mask h_(hor)=h_(ver) ^(t).

7.2 Bilateral Filter (BF)

Bilateral filter is proposed in JVET-L0406, and it is always applied toluma blocks with non-zero transform coefficients and slice quantizationparameter larger than 17. Therefore, there is no need to signal theusage of the bilateral filter. Bilateral filter, if applied, isperformed on decoded samples right after the inverse transform. Inaddition, the filter parameters, i.e., weights are explicitly derivedfrom the coded information.

The filtering process is defined as:

P′ _(0,0) =P _(0,0)+Σ_(k=1) ^(K) W _(k)(abs(P _(k,0) −P _(0,0)))×(P_(k,0) −P _(0,0)).  (1)

Herein, P_(0,0) is the intensity of the current sample and P′_(0,0) isthe modified intensity of the current sample, P_(k,0) and W_(k) are theintensity and weighting parameter for the k-th neighboring sample,respectively. An example of one current sample and its four neighboringsamples (i.e., K=4) is depicted in FIG. 5.

More specifically, the weight W_(k)(x) associated with the k-thneighboring sample is defined as follows:

$\begin{matrix}{{{{W_{k}(x)} = {Distance_{k} \times Rang{{e_{k}(x)}.{Herein}}}},{{Distance}_{k} = {{e^{({- \frac{10000}{2\sigma_{d}^{2}}})}/1} + {4*e^{({- \frac{10000}{2\sigma_{d}^{2}}})}\mspace{14mu}{and}}}}}\mspace{14mu}{{{Range}_{k}(x)} = {e^{({- \frac{x^{2}}{8*{({{QP}17})}*{({{QP}17})}}})}.}}} & (2)\end{matrix}$

Herein, σ_(d) is dependent on the coded mode and coding block sizes. Thedescribed filtering process is applied to intra-coded blocks, andinter-coded blocks when TU is further split, to enable parallelprocessing.

To better capture statistical properties of video signal, and improveperformance of the filter, weights function resulted from Equation (2)are being adjusted by the σ_(d) parameter, tabulated in Table 4 as beingdependent on coding mode and parameters of block partitioning (minimalsize).

TABLE 4 Value of σ_(d) d for different block sizes and coding modesMin(block width, block height) Intra mode Inter mode 4 82 62 8 72 52Other 52 32

To further improve the coding performance, for inter-coded blocks whenTU is not split, the intensity difference between current sample and oneof its neighboring samples is replaced by a representative intensitydifference between two windows covering current sample and theneighboring sample. Therefore, the equation of filtering process isrevised to:

$\begin{matrix}{P_{0,0}^{\prime} = {P_{0,0} + {\sum_{k = 1}^{N}{{W_{k}\left( {\frac{1}{M}{\sum_{m = {{- M}/2}}^{M/2}{a\;{{bs}\left( {P_{k,m} - P_{0,m}} \right)}}}} \right)} \times \left( {P_{k,0} - P_{0,0}} \right)}}}} & (4)\end{matrix}$

Herein, P_(k,m) and P_(0,m) represent the m-th sample value within thewindows centered at P_(k,0) and P_(0,0), respectively. In this proposal,the window size is set to 3×3. An example of two windows coveringP_(2,0) and P_(0,0) are depicted in FIG. 6.

7.3 Hadamard Transform Domain Filter (HF)

In JVET-K0068, in-loop filter in 1D Hadamard transform domain which isapplied on CU level after reconstruction and has multiplication freeimplementation. Proposed filter is applied for all CU blocks that meetthe predefined condition and filter parameters are derived from thecoded information.

Proposed filtering is always applied to luma reconstructed blocks withnon-zero transform coefficients, excluding 4×4 blocks and if slicequantization parameter is larger than 17. The filter parameters areexplicitly derived from the coded information. Proposed filter, ifapplied, is performed on decoded samples right after inverse transform.

For each pixel from reconstructed block pixel processing comprises thefollowing steps:

-   -   Scan 4 neighboring pixels around processing pixel including        current one according to scan pattern    -   4 point Hadamard transform of read pixels    -   Spectrum filtering based on the following formula:

$\begin{matrix}{{{F\left( {i,\ \sigma} \right)} = {\frac{{R(i)}^{2}}{{R(i)}^{2} + \sigma^{2}}*R}}(i)} & \;\end{matrix}$

Herein, (i) is index of spectrum component in Hadamard spectrum, R(i) isspectrum component of reconstructed pixels corresponding to index, a isfiltering parameter deriving from codec quantization parameter QP usingfollowing equation:

σ=2^((1+0.126*(QP−27))).

The example of scan pattern is shown in FIG. 7, wherein A is the currentpixel and {B,C,D} are surrounding pixels.

For pixels laying on CU boundary, the scan pattern is adjusted ensuringall required pixels are within current CU.

8 Virtual Pipelining Data Units (VPDU)

Virtual pipeline data units (VPDUs) are defined as non-overlappingM×M-luma(L)/N×N-chroma(C) units in a picture. In hardware decoders,successive VPDUs are processed by multiple pipeline stages at the sametime; different stages process different VPDUs simultaneously. The VPDUsize is roughly proportional to the buffer size in most pipeline stages,so it is said to be very important to keep the VPDU size small. In HEVChardware decoders, the VPDU size is set to the maximum transform block(TB) size. Enlarging the maximum TB size from 32×32−L/16×16−C(as inHEVC) to 64×64−L/32×32−C (as in the current VVC) can bring coding gains,which results in 4× of VPDU size (64×64−L/32×32−C) expectedly incomparison with HEVC. However, in addition to quadtree (QT) coding unit(CU) partitioning, ternary tree (TT) and binary tree (BT) are adopted inVVC for achieving additional coding gains, and TT and BT splits can beapplied to 128×128−L/64×64−C coding tree blocks (CTUs) recursively,which is said to lead to 16× of VPDU size (128×128−L/64×64−C) incomparison with HEVC.

In current design of VVC, the VPDU size is defined as 64×64−L/32×32−C.

9 Drawbacks of Existing Implementations

The non-linear ALF (NLALF) design in JVET-N0242 has the followingproblems:

-   -   (1) The classification process in GALF relies on the gradient        and Laplacian activities which utilize the reconstructed samples        before applying ALF. However, the classification results may be        inaccurate. For example, for one sample, differences between it        and its neighbors may be quite similar while for another sample,        difference between it and one neighbor may be too different and        for all others, difference may be too small. For these two        cases, they may be classified to one class index which may be        unreasonable.    -   (2) The clipping parameters are associated with the filter        coefficient. However, one filter may be utilized for multiple        classes due to filter merging process. In addition, for two        blocks with the same class index (0 . . . 24 in current GALF        design), the filter coefficients and clipping parameters are the        same. However, the two blocks may have different        characteristics, for example, the selected geometric        transformation may be different. Using same clipping parameters        may be suboptimal.    -   (3) The index of clipping parameters is signaled per non-zero        filter coefficient which requires to construct the filter        coefficients in the parsing stage. Such a design is undesirable        for hardware implementation.    -   (4) In the temporal prediction process, one block may inherit        the filter coefficients from previously coded frames. How to        handle the clipping parameters need to be studied.    -   (5) The clipping function K(d, b)=min(b, max(−b, d)) with b as        the upper bounder and −b as the lower bound, d as the input. The        restriction of equal magnitudes for the upper and lower bounder        may be suboptimal.

10 Exemplary Methods for Temporal Prediction of Parameters in Non-LinearALF

Embodiments of the presently disclosed technology overcome the drawbacksof existing implementations, thereby providing video coding with highercoding efficiencies. Temporal prediction of parameters in non-linearadaptive loop filtering, based on the disclosed technology, may enhanceboth existing and future video coding standards, is elucidated in thefollowing examples described for various implementations. The examplesof the disclosed technology provided below explain general concepts, andare not meant to be interpreted as limiting. In an example, unlessexplicitly indicated to the contrary, the various features described inthese examples may be combined.

In these examples, one filter may be associated with multiple filtercoefficients. One set of filters represents multiple filters. Denote thei-th filter by F^(i) and its associated filter coefficients by F^(i)k,such as the variable k denotes the k-th filter coefficient associatedwith F^(i), e.g., it may be corresponding to Ck in FIG. 2.

-   -   1. It is proposed to determine the NLALF parameters (e.g.,        on/off control flag, clipping parameters) according to coded        information.        -   a. The NLALF parameters (e.g., on/off control flag, clipping            parameters) may be dependent on coded mode information.            -   i. In one example, which NLALF parameters to be selected                may be determined by the coded mode, such as intra or                non-intra mode.            -   ii. In one example, which NLALF parameters to be                selected may be determined by the coded mode, such as                intra or inter mode.            -   iii. In one example, which NLALF parameters to be                selected may be determined by the coded mode, such as                IBC or non-IBC mode.        -   b. The NLALF parameters (e.g., on/off control flag, clipping            parameters) may be dependent on transform information.            -   i. In one example, they may be dependent on whether                transform skip is applied or not.        -   c. The NLALF parameters (e.g., on/off control flag, clipping            parameters) may be dependent on residual information.            -   i. In one example, they may be dependent on whether the                block contains non-zero coefficients.        -   d. The NLALF parameters (e.g., on/off control flag, clipping            parameters) may be dependent on tile group types/picture            types.        -   e. The NLALF parameters (e.g., on/off control flag, clipping            parameters) may be dependent on temporal layer            information/reference picture information associated with            one tile/tile group/slice etc. al.            -   i. In one example, they may be dependent on whether all                reference pictures are associated with smaller POC                values compared to current picture.            -   ii. In one example, they may be dependent on whether all                reference pictures are associated with smaller or equal                POC values compared to current picture.        -   f. It is proposed to determine the NLALF parameters (e.g.,            on/off control flag, clipping parameters) according to            reference picture/motion information associated with one            block.    -   2. It is proposed to determine the NLALF parameters (e.g.,        on/obiff control flag, clipping parameters) according to the        geometric transformation.        -   a. In one example, for two M×N blocks, even they are            associated with the same filter (e.g., due to the same class            index), the associated NLALF parameters (e.g., clipping            parameters) may be different.        -   b. In one example, for one filter coefficient, indications            of more than one clipping parameter may be signaled.            -   i. In one example, how many clipping parameters/or                indices of clipping parameters or other representations                of clipping parameters may be dependent on the number of                allowed geometric transformations.            -   ii. In one example, predictive coding of clipping                parameters/indices of clipping parameters associated                with one filter parameter may be applied.                -   1) In one example, a clipping parameter of one                    filter for one sample or block may be predicted by                    another clipping parameter of another filter used                    for a spatial/temporal adjacent or non-adjacent                    neighbouring sample or block.    -   3. It is proposed that magnitudes of the upper bound and the        lower bound in the clipping function may be unequal.        -   a. In one example, indications of both upper bound and lower            bound for one clipping function may be signaled.            -   i. Alternatively, furthermore, predictive coding between                upper bound and lower bound may be applied.    -   4. It is proposed to directly code the indications of clipping        parameters (such as indices) with fixed length.        -   a. In one example, each of them may be coded with N-bits            (e.g., N is set to 2).            -   i. In one example, N may be fixed;            -   ii. In one example, N may be signaled;            -   iii. In one example, N may depend on coding information,                such as QP, picture dimensions and so on.        -   b. Alternatively, they may be coded with truncated unary            method with a maximum value N.            -   i. In one example, N may be fixed;            -   ii. In one example, N may be signaled;            -   iii. In one example, N may depend on coding information,                such as QP, picture dimensions and so on.        -   c. Alternatively, they may be coded with Exponential-Golomb            method but with fixed order for one filter/for one filter            set.        -   d. Alternatively, they may be coded with run-length coding.            -   i. In one example, for each filter, the index of a                clipping parameter may be firstly coded as ‘run’, and                the number of consecutive same clipping parameter as                ‘length’.            -   ii. In one example, for each k-th filter coefficient in                all filters, the index of a clipping parameter                associated with F′ may be firstly coded as ‘run’, and                the number of same clipping parameter in other filters                as ‘length’.        -   e. In one example, predictive coding of indications of            clipping parameters (such as indices) may be applied.            -   i. In one example, predictive coding may be applied for                clipping parameters within one filter.            -   ii. In one example, predictive coding may be applied for                clipping parameters among different filters.                -   1) In one example, predictive coding may be applied                    for clipping parameters among different filters for                    one color component.                -   2) In one example, predictive coding may be applied                    for clipping parameters among different filters for                    multiple color component.                -   3) In one example, predictive coding may be applied                    for clipping parameters of filters used for                    different samples or blocks.            -   iii. In one example, predictive coding may be applied                for clipping parameters signaled in different APSs.    -   5. It is proposed to decouple the parsing of clipping parameters        and construction of filter coefficients.        -   a. In one example, the parsing of clipping parameters (e.g.,            index of clipping parameters) is independent from the values            of filter coefficients.        -   b. In one example, when the filter coefficient is equal to            0, indication of the associated clipping parameter may be            still signaled.    -   6. When one block inherits filter coefficients from the i-th        filter, the associated clipping parameters with the i-th filter        may be not inherited.        -   a. In one example, when temporal prediction is enabled for            one block, instead of directly inheriting the associated            clipping parameters, whether to use non-local ALF or not            (applying clipping or not) may be signaled.            -   i. In one example, if it is determined to apply                clipping, the associated clipping parameters may be also                inherited.        -   b. In one example, suppose the filter coefficients are            inherited/predicted from the i-th filter, the clipping            parameters are inherited/predicted from the j-th filter, i            may be unequal to j.        -   c. In one example, suppose the filter coefficients are            inherited/predicted from the i-th filter, the clipping            parameters are inherited/predicted from the j-th filter, the            i-th and j-th filter may be associated with different filter            set.            -   i. In one example, the i-th filter may be associated                with a first picture/tile group/tile/slice and the j-th                filter may be associated with a second picture/tile                group/tile/slice.            -   ii. In one example, i is unequal to j. Alternatively, i                is equal to j.        -   d. In one example, indications of clipping parameters            associated with which filter may be signaled, such as the            filter index.        -   e. In one example, indications of clipping parameters            associated with which filter set may be signaled, such as            the APS index.            -   i. Alternatively, furthermore, the filter index may be                further signalled.    -   7. In the classification process, instead of directly using the        sample difference, the clipped sample difference may be        utilized.        -   a. In one example, in the gradient calculation process,            clipped sample difference or clipped gradients may be used.        -   b. In one example, in the activity calculation process,            clipped sample difference or clipped gradients may be used.        -   c. In one example, the following may be used to calculate            the vertical gradient:

V _(k,l)=|clip1(R(k,l)−R(k,l−1))+clip2(R(k,l)−R(k,l+1))

-   -   -    wherein clip1 and clip2 are two clipping functions.        -   d. In one example, the following may be used to calculate            the horizontal gradient:

H _(k,l)=|clip1(R(k,l)−md R(k−l,1))+clip2(R(k,l)−R(k+1,l))|

-   -   -    wherein clip1 and clip2 are two clipping functions.

    -   8. Whether to apply the clipping operations may depend on the        locations of the samples (such as I(x+i, y+j) in Section 5.2) to        be used in the filtering process.        -   a. In one example, if the sample in a filter support is not            located a CU/PU/TU/picture/tile/tile group boundary,            clipping may be disabled.        -   b. In one example, if the sample in a filter support is            located a CU/PU/TU/picture/tile/tile group/CTU/Virtual            Pipelining Data Unit (VPDU) boundary, clipping may be            applied.        -   c. Alternatively, whether to apply the clipping operations            may depend on the distance of the samples (such as I(x+i,            y+j) in Section 5.2) to be used in the filtering process            from the CU/PU/TU/picture/tile/tile group/CTU/VPDU boundary.            -   i. In one example, the distance may be pre-defined                (e.g., N-pel).            -   ii. In one example, the distance may be signaled.

    -   9. The filter shape (a.k.a. filter support) used in adaptive        loop filter process may depend on the color representation.        -   a. In one example, the filter support for all components            (such as Y, Cb, Cr) should be the same when the color format            is 4:4:4.            -   i. For example, the filter support is 7*7 diamond as                shown in FIG. 2B.            -   ii. For example, the filter support is 5*5 diamond as                shown in FIG. 2A.        -   b. In one example, the support area for all components when            the color format is RGB.            -   i. For example, the filter support is 7*7 diamond as                shown in FIG. 2B.            -   ii. For example, the filter support is 5*5 diamond as                shown in FIG. 2A.

The examples described above may be incorporated in the context of themethod described below, e.g., methods 800, 810 and 820, which may beimplemented at a video decoder or a video encoder.

FIG. 8A shows a flowchart of an exemplary method for video processing.The method 800 includes, at step 802, determining, based on acharacteristic of a current video block, one or more parameters of anon-linear filtering operation.

The method 800 includes, at step 804, performing, based on the one ormore parameters, a conversion between the current video block and abitstream representation of the current video block.

In some embodiments, the characteristic of the current video block is acoding mode of the current video block. In an example, the coding modeof the current video block is an intra mode, a non-intra mode, an intrablock copy (IBC) mode or a non-IBC mode.

In some embodiments, the characteristic is transform information. In anexample, the transform information comprises an indication of transformskip being applied to the current video block.

In some embodiments, the characteristic is residual information. In anexample, the residual information comprises zero-valued coefficients inthe current video block.

In some embodiments, the characteristic is a tile group type or apicture type of a tile group or a picture comprising the current videoblock.

In some embodiments, the characteristic is temporal layer information orreference information associated with a tile, a tile group, a picture ora slice comprising the current video block.

In some embodiments, the characteristic is a reference picture or motioninformation associated with the current video block.

In some embodiments, the characteristic is a geometric transformation.

In some embodiments, the one or more parameters comprises an on/offcontrol flag or one or more parameters of a clipping function.

In some embodiments, a magnitude of an upper bound of the clippingfunction is different from a magnitude of a lower bound of the clippingfunction. In an example, predictive coding is applied between the upperbound and the lower bound of the clipping function.

FIG. 8B shows a flowchart of an exemplary method for video processing.The method 810 includes, at step 812, configuring, for a current videoblock, one or more parameters of a clipping operation that is part of anon-linear filtering operation.

The method 810 includes, at step 814, performing, based on the one ormore parameters, a conversion between the current video block and abitstream representation of the current video block.

In some embodiments, the one or more parameters is coded with a fixedlength of N bits. In other embodiments, the one or more parameters iscoded with a truncated unary method with a maximum value of N. In anexample, N is fixed. In another example, N is signaled. In yet anotherexample, N is based on coded information of the current video block thatcomprises a quantization parameter or a dimension of the picturecomprising the current video block.

In some embodiments, the one or more parameters is coded with anExponential-Golomb method with a fixed order for one filter or filterset.

In some embodiments, the one or more parameters is coded with run-lengthcoding.

In some embodiments, the one or more parameters is signaledindependently from values of at least one filter coefficient.

In some embodiments, the one or more parameters further comprises filtercoefficients, wherein the current video block inherits the filtercoefficients from an i-th filter, and wherein the one or more parametersof the clipping function are inherited from a j-th filter that isdifferent from the i-th filter.

FIG. 8C shows a flowchart of an exemplary method for video processing.The method 820 includes, at step 822, configuring a non-linear filteringoperation that comprises a clipping operation.

The method 820 includes, at step 824, performing, based on thenon-linear filtering operation, a conversion between a current videoblock and a bitstream representation of the current video block.

In some embodiments, the method 820 further includes the step ofperforming a gradient calculation process that uses a clipped sampledifference or a clipped gradient generated using the clipping operation.In other embodiments, the method 820 further includes the step ofperforming an activity calculation process that uses a clipped sampledifference or a clipped gradient generated using the clipping operation.In an example, the clipped gradient comprises a vertical gradient thatis computed as V_(k,l)=|clip1(R(k,l)−R(k,l−1))+clip2(R(k,l)−R(k,l+1))|.In another example, the clipped gradient comprises a horizontal gradientthat is computed asH_(k,l)=|clip1(R(k,l)−R(k−1,l))+clip2(R(k,l)−R(k+1,l))|, where clip1 andclip2 are a first and a second clipping function, respectively.

In some embodiments, performing the conversion comprises filtering oneor more samples of the current video block, and an operation of theclipping operation is configured based on a location of the one or moresamples.

In some embodiments, the location of the one or more samples is aboundary of a coding unit (CU), a prediction unit (PU), a transform unit(TU), a picture, a tile, a tile group, a coding tree unit (CTU) or avirtual pipelining data unit (VPDU).

In some embodiments, and in the context of methods 800, 810 and 820, ashape of a filter used in the non-linear filtering operation is based ona color representation. In an example, the color representationcomprises a 4:4:4 color format or an RGB color format. In anotherexample, the filter is a diamond shaped filter.

In some embodiments, and in the context of methods 800, 810 and 820, thenon-linear filtering operation is a non-linear adaptive loop filteringoperation.

11 Example Implementations of the Disclosed Technology

FIG. 9 is a block diagram of a video processing apparatus 900. Theapparatus 900 may be used to implement one or more of the methodsdescribed herein. The apparatus 900 may be embodied in a smartphone,tablet, computer, Internet of Things (IoT) receiver, and so on. Theapparatus 900 may include one or more processors 902, one or morememories 904 and video processing hardware 906. The processor(s) 902 maybe configured to implement one or more methods (including, but notlimited to, methods 800, 810 and 820) described in the present document.The memory (memories) 904 may be used for storing data and code used forimplementing the methods and techniques described herein. The videoprocessing hardware 906 may be used to implement, in hardware circuitry,some techniques described in the present document.

In some embodiments, the video coding methods may be implemented usingan apparatus that is implemented on a hardware platform as describedwith respect to FIG. 9.

FIG. 10 is a block diagram showing an example video processing system1000 in which various techniques disclosed herein may be implemented.Various implementations may include some or all of the components of thesystem 1000. The system 1000 may include input 1002 for receiving videocontent. The video content may be received in a raw or uncompressedformat, e.g., 8 or 10 bit multi-component pixel values, or may be in acompressed or encoded format. The input 1002 may represent a networkinterface, a peripheral bus interface, or a storage interface. Examplesof network interface include wired interfaces such as Ethernet, passiveoptical network (PON), etc. and wireless interfaces such as Wi-Fi orcellular interfaces.

The system 1000 may include a coding component 1004 that may implementthe various coding or encoding methods described in the presentdocument. The coding component 1004 may reduce the average bitrate ofvideo from the input 1002 to the output of the coding component 1004 toproduce a coded representation of the video. The coding techniques aretherefore sometimes called video compression or video transcodingtechniques. The output of the coding component 1004 may be eitherstored, or transmitted via a communication connected, as represented bythe component 1006. The stored or communicated bitstream (or coded)representation of the video received at the input 1002 may be used bythe component 1008 for generating pixel values or displayable video thatis sent to a display interface 1010. The process of generatinguser-viewable video from the bitstream representation is sometimescalled video decompression. Furthermore, while certain video processingoperations are referred to as “coding” operations or tools, it will beappreciated that the coding tools or operations are used at an encoderand corresponding decoding tools or operations that reverse the resultsof the coding will be performed by a decoder.

Examples of a peripheral bus interface or a display interface mayinclude universal serial bus (USB) or high definition multimediainterface (HDMI) or Displayport, and so on. Examples of storageinterfaces include SATA (serial advanced technology attachment), PCI,IDE interface, and the like. The techniques described in the presentdocument may be embodied in various electronic devices such as mobilephones, laptops, smartphones or other devices that are capable ofperforming digital data processing and/or video display.

FIG. 11 shows a flowchart of an example method for visual mediaprocessing. Steps of this flowchart are discussed in connection withexample 4 in Section 10 of this document. At step 1102, the processconfigures, for a current video block, one or more parameters of aclipping operation that is part of a non-linear filtering operation. Atstep 1104, the process performs, based on the one or more parameters, aconversion between the current video block and a bitstreamrepresentation of the current video block, wherein the one or moreparameters is coded in accordance with a rule.

FIG. 12 shows a flowchart of an example method for visual mediaprocessing. Steps of this flowchart are discussed in connection withexample 1 in Section 10 of this document. At step 1202, the processdetermines, based on a characteristic of a current video block, one ormore parameters of a non-linear filtering operation. At step 1204, theprocess performs, based on the one or more parameters, a conversionbetween the current video block and a bitstream representation of thecurrent video block.

FIG. 13 shows a flowchart of an example method for visual mediaprocessing. Steps of this flowchart are discussed in connection withexample 5 in Section 10 of this document. At step 1302, the processconfigures, for a current video block, one or more parameters of aclipping operation that is part of a non-linear filtering operation. Atstep 1304, the process performs, based on the one or more parameters, aconversion between the current video block and a bitstreamrepresentation of the current video block, wherein the one or moreparameters are presented in the bitstream representation independentlyfrom values of at least one filter coefficient associated with thenon-linear filtering operation.

FIG. 14 shows a flowchart of an example method for visual mediaprocessing. Steps of this flowchart are discussed in connection withexample 6 in Section 10 of this document. At step 1402, the processconfigures, for a current video block, one or more parameters of aclipping operation that is part of a non-linear filtering operation. Atstep 1404, the process performs, based on the one or more parameters, aconversion between the current video block and a bitstreamrepresentation of the current video block, wherein the current videoblock inherits filter coefficients from an i-th filter, and wherein afirst rule associated with inheritance of the one or more parameters ofthe clipping operation is different from a second rule associated withinheritance of the filter coefficients.

Various embodiments discussed herein are now presented in clause-basedformat.

A1. A method for visual media processing, comprising:

-   -   configuring, for a current video block, one or more parameters        of a clipping operation that is part of a non-linear filtering        operation; and    -   performing, based on the one or more parameters, a conversion        between the current video block and a bitstream representation        of the current video block, wherein the one or more parameters        is coded in accordance with a rule.

A2. The method of clause A1, wherein the rule specifies coding the oneor more parameters with a fixed length of N bits.

A3. The method of clause A1, wherein the rule specifies coding the oneor more parameters with a truncated unary method based on a maximumvalue of N.

A4. The method of any one or more of clauses A1-A3, wherein N is fixed.

A5. The method of any one or more of clauses A1-A3, wherein N issignaled in the bitstream representation.

A6. The method of any one or more of clauses A1-A3, wherein N is basedon coded information of the current video block that comprises aquantization parameter or a dimension of the picture comprising thecurrent video block.

A7. The method of clause A1, wherein the non-linear filtering operationis based on a filter, wherein the rule specifies coding the one or moreparameters with an Exponential-Golomb method with a fixed order for onefilter or filter set.

A8. The method of clause A1, wherein the rule specifies coding the oneor more parameters based on a run-length coding method.

A9. The method of clause A5, wherein the rule further specifies that arun of the run-length coding method corresponds to an index of the oneor more parameters and a length of the run-length coding methodcorresponds to a number of consecutive parameters of the one or moreparameters that are same.

A10. The method of clause A1, wherein the rule specifies coding the oneor more parameters based on predictive coding.

A11. The method of clause A10, wherein the predictive coding is appliedfor the one or more parameters within one filter.

A12. The method of clause A10, wherein the predictive coding is appliedfor the one or more parameters among different filters.

A13. The method of clause A12, wherein the predictive coding is appliedfor the one or more parameters among different filters used for onecolor component.

A14. The method of clause A12, wherein the predictive coding is appliedfor the one or more parameters among different filters used fordifferent color components.

A15. The method of clause A12, wherein the predictive coding is appliedfor the one or more parameters among different filters used fordifferent samples of the current video block.

A16. The method of clause A12, wherein the predictive coding is appliedfor the one or more parameters among different filters used fordifferent video blocks.

A17. The method of clause A11, wherein the one or more parameters areincluded as fields in different adaptive parameters sets (APSs).

A18. The method of any one or more of clauses A1 to A17, wherein thenon-linear filtering operation is an adaptive loop filter (ALF)operation which comprises determining a filter index based on gradientcalculations in different directions.

A19. The method of any one or more of clauses A1 to A17, wherein the oneor more parameters includes a clipping index.

B1. A method for visual media processing, comprising:

-   -   determining, based on a characteristic of a current video block,        one or more parameters of a non-linear filtering operation; and    -   performing, based on the one or more parameters, a conversion        between the current video block and a bitstream representation        of the current video block.

B2. The method of clause B1, wherein the characteristic of the currentvideo block is a coding mode of the current video block.

B3. The method of clause B2, wherein the coding mode of the currentvideo block is an intra mode, a non-intra mode, an intra block copy(IBC) mode or a non-IBC mode.

B4. The method of clause B1, wherein the characteristic is transforminformation.

B5. The method of clause B4, wherein the transform information comprisesan indication of transform skip being applied to the current videoblock.

B6. The method of clause B1, wherein the characteristic is residualinformation.

B7. The method of clause B6, wherein the residual information compriseszero-valued coefficients in the current video block.

B8. The method of clause B1, wherein the characteristic is a tile grouptype or a picture type of a tile group or a picture comprising thecurrent video block.

B9. The method of clause B1, wherein the characteristic is temporallayer information or reference information associated with a tile, atile group, a picture or a slice comprising the current video block.

B10. The method of clause B1, wherein the characteristic is a referencepicture or motion information associated with the current video block.

B11. The method of clause B1, wherein the characteristic is a geometrictransformation.

B12. The method of any one or more of clauses B1 to B11, wherein the oneor more parameters comprises an on/off control flag and/or one or moreparameters of a clipping function.

B13. The method of clause B12, wherein a magnitude of an upper bound ofthe clipping function is different from a magnitude of a lower bound ofthe clipping function.

B14. The method of clause B13, wherein predictive coding is appliedbetween the upper bound and the lower bound of the clipping function.

B15. The method of clause B12, wherein the upper bound of the clippingfunction and the lower bound of the clipping function are included asfields in the bitstream representation.

B16. The method of clause B10, wherein the non-linear filteringoperation includes use of a first filter and a second filter, andwherein the one or more parameters of the second filter are predictedusing the one or more parameters of the first filter.

B17. The method of clause B10, wherein the first filter and the secondfilter are applied on different sets of samples of the current videoblock.

B18. The method of clause B10, wherein the first filter and the secondfilter are applied on samples associated with different video blocks.

B19. The method of any one or more of clauses A1 to B18, wherein a shapeof a filter used in the non-linear filtering operation is based on acolor representation of a sample associated with the current videoblock.

B20. The method of clause B19, wherein the color representationcomprises a 4:4:4 color format or an RGB color format.

B21. The method of clause B19, wherein the filter is a diamond shapedfilter.

B22. The method of clause B19, wherein the diamond shaped filter is ofsize 5×5 or 7×7.

B23. The method of any one or more of clauses B1 to B22, wherein thenon-linear filtering operation is a non-linear adaptive loop filteringoperation.

B24. The method of any one or more of clauses B1 to B23, wherein thenon-linear filtering operation is an adaptive loop filter (ALF)operation which comprises determining a filter index based on gradientcalculations in different directions.

B25. The method of any one or more of clauses B1 to B23, wherein the oneor more parameters includes a clipping index.

B26. The method of any of clauses A1-B25, wherein the conversionincludes generating the bitstream representation from the current videoblock.

B27. The method of any of clauses A1-B25, wherein the conversionincludes generating pixel values of the current video block from thebitstream representation.

B28. A video encoder apparatus comprising a processor configured toimplement a method recited in any one or more of clauses A1-B25.

B29. A video decoder apparatus comprising a processor configured toimplement a method recited in any one or more of clauses A1-B25.

B30. A computer readable medium having code stored thereon, the codeembodying processor-executable instructions for implementing a methodrecited in any of or more of clauses A1-B25.

C1. A method for visual media processing, comprising:

-   -   configuring, for a current video block, one or more parameters        of a clipping operation that is part of a non-linear filtering        operation; and    -   performing, based on the one or more parameters, a conversion        between the current video block and a bitstream representation        of the current video block, wherein the one or more parameters        are presented in the bitstream representation independently from        values of at least one filter coefficient associated with the        non-linear filtering operation.

C2. The method of clause C1, wherein the one or more parameters arepresented in the bitstream representation in case that a value of the atleast one filter coefficient is zero.

C3. The method of any one or more of clauses C1-C2, wherein the one ormore parameters are presented in the bitstream representation regardlessof the value of the at least one filter coefficient.

C4. The method of any one or more of clauses C1 to C3, wherein thenon-linear filtering operation is an adaptive loop filter (ALF)operation which comprises determining a filter index based on gradientcalculations in different directions.

C5. The method of any one or more of clauses C1 to C3, wherein the oneor more parameters includes a clipping index.

D1. A method for visual media processing, comprising:

-   -   configuring, for a current video block, one or more parameters        of a clipping operation that is part of a non-linear filtering        operation; and    -   performing, based on the one or more parameters, a conversion        between the current video block and a bitstream representation        of the current video block, wherein the current video block        inherits filter coefficients from an i-th filter, and wherein a        first rule associated with inheritance of the one or more        parameters of the clipping operation is different from a second        rule associated with inheritance of the filter coefficients.

D2. The method of clause D1, wherein the first rule specifies excludinginheritance of the one or more parameters of the clipping operation fromthe i-th filter.

D3. The method of clause D1, further comprising:

upon identifying that temporal prediction is enabled for the currentvideo block, making a determination of whether to apply or disable theclipping operation.

D4. The method of clause D1, further comprising:

-   -   upon identifying that temporal prediction is enabled for the        current video block, making a determination of whether to apply        or exclude inheritance of the one or more parameters of the        clipping operation.

D5. The method of clause D1, wherein the first rule specifies inheritingthe one or more parameters of the clipping operation from a j-th filter.

D6. The method of clause D1, wherein the first rule specifies inheritingthe one or more parameters of the clipping operation from the j-thfilter, and wherein the j-th filter and the i-th filter are associatedwith different filter sets.

D7. The method of clause D6, wherein the j-th filter and the i-th filterare associated with different pictures and/or tile groups and/or tilesand/or slices.

D8. The method of clause D6, wherein the j-th filter and the i-th filterare same.

D9. The method of clause D5, wherein the j-th filter and the i-th filterare different.

D10. The method of clause D1, wherein the first rule specifies includingthe one or more parameters of the clipping operation as fields in thebitstream representation.

D11. The method of clause D10, wherein the fields include an adaptiveparameter set (APS) index.

D12. The method of any one or more of clauses D1-D11, wherein theclipping operation includes computing a clipped sample difference or aclipping gradient.

D13. The method of clause D12, wherein the clipped gradient comprises avertical gradient that is computed as

V_(k,l)=|clip1(R(k,l)−R(k,l−1))+clip2(R(k,l)−R(k,l+1))|,

-   -   wherein clip1 and clip2 are a first and a second clipping        function, respectively and R(i, j) denotes a sample of the        current video block.

D14. The method of clause D12, wherein the clipped gradient comprises ahorizontal gradient that is computed as

H_(k,l)=|clip1(R(k,l)−R(k−1,l))+clip2(R(k,l)−R(k+1,l))|,

-   -   wherein clip1 and clip2 are a first and a second clipping        function, respectively and R(i, j) denotes a sample of the        current video block.

D15. The method of any one or more of clauses D1 to D14, furthercomprising:

-   -   making a determination, based on a location of a sample used in        the filtering operation, of whether to selectively enable or        disable the clipping operation.

D16. The method of clause D15, wherein the clipping operation isdisabled if the sample is not located at a boundary of one or more ofthe following: a coding unit, a partition unit, a transform unit, apicture, a tile, or a tile group.

D17. The method of clause D15, wherein the clipping operation is enabledif the sample is located at a boundary of one or more of the following:a coding unit, a partition unit, a transform unit, a picture, a tile, atile group, a coding tree unit, or a virtual pipelining data unit.

D18. The method of clause D15, wherein the location is with respect to adistance between the sample and a boundary of one or more of thefollowing: a coding unit, a partition unit, a transform unit, a picture,a tile, a tile group, a coding tree unit, or a virtual pipelining dataunit.

D19. The method of clause D18, wherein the distance is pre-defined.

D20. The method of clause D18, wherein the distance is signaled in thebitstream representation.

D21. The method of any one or more of clauses D1 to D20, wherein a shapeof a filter used in the non-linear filtering operation is based on acolor representation of a sample associated with the current videoblock.

D22. The method of clause D21, wherein the color representationcomprises a 4:4:4 color format or an RGB color format.

D23. The method of clause D21, wherein the filter is a diamond shapedfilter.

D24. The method of clause D23, wherein the diamond shaped filter is ofsize 5×5 or 7×7.

D25. The method of any one or more of clauses D1 to D24, wherein thenon-linear filtering operation is an adaptive loop filtering (ALF)operation which comprises determining a filter index based on gradientcalculations in different directions.

D26. The method of any one or more of clauses D1 to D24, wherein the oneor more parameters includes a clipping index.

D27. The method of any of clauses C1-D26, wherein the conversionincludes generating the bitstream representation from the current videoblock.

D28. The method of any of clauses C1-D26, wherein the conversionincludes generating pixel values of the current video block from thebitstream representation.

D29. A video encoder apparatus comprising a processor configured toimplement a method recited in any one or more of clauses C1-D26.

D30. A video decoder apparatus comprising a processor configured toimplement a method recited in any one or more of clauses C1-D26.

D31. A computer readable medium having code stored thereon, the codeembodying processor-executable instructions for implementing a methodrecited in any of or more of clauses C1-D26.

In the present document, the term “video processing” or “visual mediaprocessing” or “processing of visual media” may refer to video encoding,video decoding, video compression or video decompression. For example,video compression algorithms may be applied during conversion from pixelrepresentation of a video to a corresponding bitstream representation orvice versa. The bitstream representation of a current video block may,for example, correspond to bits that are either co-located or spread indifferent places within the bitstream, as is defined by the syntax. Forexample, a macroblock may be encoded in terms of transformed and codederror residual values and also using bits in headers and other fields inthe bitstream. Furthermore, during conversion, a decoder may parse abitstream with the knowledge that some fields may be present, or absent,based on the determination, as is described in the above solutions.Similarly, an encoder may determine that certain syntax fields are orare not to be included and generate the coded representation accordinglyby including or excluding the syntax fields from the codedrepresentation.

From the foregoing, it will be appreciated that specific embodiments ofthe presently disclosed technology have been described herein forpurposes of illustration, but that various modifications may be madewithout deviating from the scope of the invention. Accordingly, thepresently disclosed technology is not limited except as by the appendedclaims.

Implementations of the subject matter and the functional operationsdescribed in this patent document can be implemented in various systems,digital electronic circuitry, or in computer software, firmware, orhardware, including the structures disclosed in this specification andtheir structural equivalents, or in combinations of one or more of them.Implementations of the subject matter described in this specificationcan be implemented as one or more computer program products, i.e., oneor more modules of computer program instructions encoded on a tangibleand non-transitory computer readable medium for execution by, or tocontrol the operation of, data processing apparatus. The computerreadable medium can be a machine-readable storage device, amachine-readable storage substrate, a memory device, a composition ofmatter effecting a machine-readable propagated signal, or a combinationof one or more of them. The term “data processing unit” or “dataprocessing apparatus” encompasses all apparatus, devices, and machinesfor processing data, including by way of example a programmableprocessor, a computer, or multiple processors or computers. Theapparatus can include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, and it can bedeployed in any form, including as a stand-alone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Computer readable media suitable for storingcomputer program instructions and data include all forms of nonvolatilememory, media and memory devices, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

It is intended that the specification, together with the drawings, beconsidered exemplary only, where exemplary means an example. As usedherein, the singular forms “a”, “an” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. Additionally, the use of “or” is intended to include“and/or”, unless the context clearly indicates otherwise.

While this patent document contains many specifics, these should not beconstrued as limitations on the scope of any invention or of what may beclaimed, but rather as descriptions of features that may be specific toparticular embodiments of particular inventions. Certain features thatare described in this patent document in the context of separateembodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Moreover, the separation of various system components in theembodiments described in this patent document should not be understoodas requiring such separation in all embodiments.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this patent document.

What is claimed is:
 1. A method of processing video data, comprising: determining, that a non-linear filtering operation is applied for a current video region of a video; generating at least one first filtering index for the current video region; deriving a first filtering coefficient set based on the at least one first filtering index and at least one coefficient parameter syntax element; deriving a first clipping parameter set based on the at least one first filtering index and at least one filtering clipping syntax element; performing the non-linear filtering operation based on the first filtering coefficient set and the first clipping parameter set; and performing a conversion between the current video region and a bitstream of the video; wherein the coefficient parameter syntax element is present in the bitstream independently from values of the first filtering coefficient set.
 2. The method of claim 1, wherein the at least one filtering clipping syntax element is present in the bitstream in case that at least one value of the first filtering coefficient set is zero.
 3. The method of claim 1, wherein the at least one filtering clipping syntax element is present in the bitstream regardless of the values of the first filtering coefficient set.
 4. The method of claim 1, wherein the at least one filtering clipping syntax element and the at least one coefficient parameter syntax element are present in a same adaptive parameter set.
 5. The method of claim 4, wherein the at least one filtering clipping syntax element includes a clipping index.
 6. The method of claim 4, wherein the at least one coefficient parameter syntax element includes a coefficient absolute value.
 7. The method of claim 4, wherein the at least one filtering clipping syntax element is coded with a fixed length of 2 bits and the at least one coefficient parameter syntax element is coded using Exponential-Golomb model with fixed order.
 8. The method of claim 1, wherein the current video region includes a coding tree block or a slice.
 9. The method of claim 1, wherein the at least one first filtering index is derived based on multiple sample differences in different directions.
 10. The method of claim 9, wherein the current video region is split into multiple M*M video sub-region, and the multiple sample differences in different directions are derived for every M*M video sub-region, and wherein M is equal to 2 or
 4. 11. The method of claim 10, wherein the multiple sample differences in different directions are derived based on 1:N subsampling rate, wherein N is great than
 1. 12. The method of claim 1, wherein the conversion includes encoding the current video region into the bitstream.
 13. The method of claim 1, wherein the conversion includes decoding the current video region from the bitstream.
 14. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to: determine, that a non-linear filtering operation is applied for a current video region of a video; generate at least one first filtering index for the current video region; derive a first filtering coefficient set based on the at least one first filtering index and at least one coefficient parameter syntax element; derive a first clipping parameter set based on the at least one first filtering index and at least one filtering clipping syntax element; perform the non-linear filtering operation based on the first filtering coefficient set and the first clipping parameter set; and perform a conversion between the current video region and a bitstream of the video; wherein the coefficient parameter syntax element is present in the bitstream independently from values of the first filtering coefficient set.
 15. The apparatus of claim 14, wherein the at least one filtering clipping syntax element is present in the bitstream in case that at least one value of the first filtering coefficient set is zero.
 16. The apparatus of claim 14, wherein the at least one filtering clipping syntax element is present in the bitstream regardless of the values of the first filtering coefficient set.
 17. A non-transitory computer-readable storage medium storing instructions that cause a processor to: determine, that a non-linear filtering operation is applied for a current video region of a video; generate at least one first filtering index for the current video region; derive a first filtering coefficient set based on the at least one first filtering index and at least one coefficient parameter syntax element; derive a first clipping parameter set based on the at least one first filtering index and at least one filtering clipping syntax element; perform the non-linear filtering operation based on the first filtering coefficient set and the first clipping parameter set; and perform a conversion between the current video region and a bitstream of the video; wherein the coefficient parameter syntax element is present in the bitstream independently from values of the first filtering coefficient set.
 18. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises: determining, that a non-linear filtering operation is applied for a current video region of a video; generating at least one first filtering index for the current video region; deriving a first filtering coefficient set based on the at least one first filtering index and at least one coefficient parameter syntax element; deriving a first clipping parameter set based on the at least one first filtering index and at least one filtering clipping syntax element; performing the non-linear filtering operation based on the first filtering coefficient set and the first clipping parameter set; and generating a bitstream of the video based on the non-linear filtering operation; wherein the coefficient parameter syntax element is present in the bitstream independently from values of the first filtering coefficient set.
 19. The non-transitory computer-readable recording medium of claim 18, wherein the at least one filtering clipping syntax element is present in the bitstream in case that at least one value of the first filtering coefficient set is zero.
 20. The non-transitory computer-readable recording medium of claim 18, wherein the at least one filtering clipping syntax element is present in the bitstream regardless of the values of the first filtering coefficient set. 